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(54) Method and apparatus for correcting degradation of book center image by image processing 
based on contour of book center image 

(57) An image processing apparatus corrects deg- 
radation of an image of a pair of spread pages of a 
bound book placed on a document table. The image 
processing apparatus includes a height obtaining unit 
which obtains a plurality of heights, above the document 
table, of a plurality of points of the pair of spread pages, 
based on a contour of the pair of spread pages which 
appears in the image, and an image correcting unit 
which corrects the degradation of the image based on 
the plurality of heights obtained by the height obtaining 
unit. The degradation may be unevenness of lightness, 
deformation, a blur, or the like. 
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Description 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

[0001] The present invention relates to an image processing method and an image processing apparatus for cor- 
recting unevenness of lightness, deformation, a blur, or the like, which occurs in an image of a center portion of a pair 
of spread (open) pages of a bound book (as an original document) when the spread pages of the book are read by a 
10 flat bed scanner (as an image reading apparatus) having a document table. The flat bed scanner is contained in gen- 
eral-purpose scanners, filing systems, copiers, facsimile machines, and various printing machines. In this specification, 
a center portion of a pair of spread pages of a bound book, i.e., a portion near the binding of a book, may be referred 
to as a book center. 

15 Description of the Related Art 

[0002] Fig. 1 is a perspective view illustrating relative positions of a document table, a book document, and an 
. image reading apparatus. When a pair of spread (open) pages of a book document is read by a flat bed scanner, the 
book center, i.e., a portion near the binding of the book, is lifted from the document table. Therefore, the image of the 
20 book center is darkened, deformed, and blurred. That is, the book center portion of the scanned image of the book doc- 
ument becomes hard to see. 

[0003] In order to solve the above problem, Japanese Unexamined Patent Publication No. 62(1 987)- 143557 dis- 
closes a document reading apparatus which comprises a distance sensor for detecting a distance between the docu- 
ment table and each position of a surface of an original document, and the sub-scanning pitch is varied accordin g to 

25 the distance between the document table and each position of the surface of the original document, so as to eliminate 
deformation in the sub-scanning direction and enable accurate reading of image information on the original document. 
[0004] in addition, Japanese Unexamined Patent Publication No 3(1 991 )-1 17965 discloses a document reading 
apparatus which comprises a document reading means arranged at a predetermined distance from an original docu- 
ment. The document reading apparatus detects curvature of surfaces of the original document, and corrects image data 

30 according to the detected curvature, when reproducing the image. 

[0005] However, the technique of Japanese Unexamined Patent Publication No. 62(1 987)- 143557 has a drawback 
that the image reading apparatus must have the distance sensor and a mechanism for changing the sub-scanning pitch 
according to the distance between the document table and each position of the surface of the original document during 
the scanning operation. On the other hand, in the document reading apparatus of Japanese Unexamined Patent Pub- 

35 lication No. 3( 1 991 )-1 17965, the document reading means is arranged to read the original document from above, and 
the document reading means must be held at a predetermined distance from the original document. That is, the docu- 
ment reading apparatus disclosed in Japanese Unexamined Patent Publication No. 3(1 991)- 11 7965 is not a flat bed 
scanner. That is, the constructions of the document reading apparatuses disclosed in Japanese Unexamined Patent 
Publication Nos. 62(1 987)-1 43557 and 3(1 991)- 117965 are different from the conventional general-purpose scanner. 

40 Therefore, the cost is considerably increased for the above special constructions. 

[0006] Further, when spread (open) pages of a book document are read by a flat bed scanner, outlines of pages 
other than the open pages appear in the scanned image as illustrated in Fig. 2, in addition to the aforementioned dark- 
ened, deformed, and blurred image of the book center. Therefore, it is difficult to determine the edges (contour) of the 
open pages and the heights of the edges by the document reading apparatuses disclosed in Japanese Unexamined 

45 Patent Publications Nos. 62(1987)-1 43557 and 3(1991)-1 17965. 

SUMMARY OF THE INVENTION 

[0007] A first object of the present invention is to provide an image processing method and an image processing 
so apparatus which can correct deformation, unevenness of lightness, and a blur in an image of a book center, using only 
tow cost image processing, where the image of the book center is obtained by scanning a pair of spread (open) pages 
of a bound book placed on a document table. 

[0008] A second object of the present invention is to provide a book-center-height obtaining unit which can obtain 
a plurality of heights (lifts) of a plurality of points of the book center above a document table, based on only an image of 
55 the book center, when the image of the book center is obtained by scanning a pair of spread pages of a bound book 
placed on a document table. 

[0009] According to the first aspect of the present invention, there is provided an image processing method which 
includes the steps of (a) obtaining a plurality of heights, above a document table, of a plurality of points of a pair of 
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spread pages of a bound book as an original document, based on a contour of the pair of spread pages which appears 
in an image of the pair of spread pages, where the bound book is placed on the document table; and (b) correcting deg- 
radation of the image of the pair of spread pages, based on the plurality of heights obtained by the height obtaining unit. 
[0010] According to the second aspect of the present invention, there is provided an image processing apparatus 
5 which contains a height obtaining unit which obtains a plurality of heights, above a document table, of a plurality of 
points of a pair of spread pages of a bound book as an original document, based on a contour of the pair of spread 
pages which appears in an image of the pair of spread pages, where the bound book is placed on the document table; 
and an image correcting unit which corrects degradation of the image of the pair of spread pages, based on the plurality 
of heights obtained by the height obtaining unit. 
10 [001 1 ] With the image processing apparatus according to the second aspect of the present invention, the heights 
of a plurality of points of the book center are obtained based on the contour of the book center in the image obtained 
by scanning the book center, and degradation of the image is corrected based on the heights obtained from the contour 
of the book center in the image. That is, the degradation of the image can be corrected based on only the image. There- 
fore, it is unnecessary to provide any specific sensor or additional device, or to change the construction of the conven- 
ts tional scanner. Thus, the degradation of the image can be corrected at low cost. 

[0012] In the image processing apparatus according to the second aspect of the present invention, the above 
height obtaining unit may contain a reference line extracting unit which extracts at least one reference line based on a 
top and bottom portions of the contour of the pair of spread pages which appears in the image; a correction area deter- 
mining unit which determines as a correction area an area of the image corresponding to a portion of the pair of spread 
20 pages which is lifted from the document table, based on the at least one reference line and the contour; a distance 
obtaining unit which obtains a distance from each pixel of a portion of the contour in at least one edge of the correction 
area, to the at least one reference line; an edge height obtaining unit which obtains a height of each point of a top and 
bottom edges of the portion of the pair of spread pages above the document table, based on the distance obtained by 
the distance obtaining unit; and a height distribution obtaining unit which obtains a height of each point of the portion of 
25 the pair of spread pages above the document table, based on the height of each point of the top and bottom edges of 
the portion o1 the pair of spread pages. 

[0013] That is. the height of each point of the edges of the book center is obtained based on the top and bottom 
portions of the contour of the book center which appears in the image, and the height of each point of the book center 
can be obtained based on the height of each point of the edges of the book center. 
30 [0014] In the above height obtaining unit, the at least one reference line is a straight line obtained by extending an 
undefbrmed straight portion of the contour of the pair of spread pages which appears in the image. 
[001 5] The height of each point of the edges of the book center can be obtained even if the spread pages in the pair 
are differently skewed. 

[001 6] When the skew angles of the spread pages in the pair are identical, it is possible to obtain the distance from 

35 each pixel of a portion of the contour in at least one edge of the correction area to only one reference line. However, 
when obtaining the distance based on only one reference line, it is necessary to provide a means to equalize the skew 
angles of the spread pages in the pair. The skew angles of the spread pages on the document table are generally dif- 
ferent, and the skew angles are often different even between the top and bottom edges of each of the spread pages. 
Therefore, when the above means for equalizing the skew angles of the spread pages is not provided, it is preferable to 

40 obtain reference lines (four in total) from the top and bottom edges of the respective spread pages in the pair, to accu- 
rately obtain the heights of the respective points of the book center above the document table. 
[0017] The correction area may be defined as an area encircled by the top and bottom portions of the contour of 
the spread pages and lines connecting top and bottom branch points of the respective spread pages, where the top and 
bottom branch points are points at which the corresponding reference lines branch from the top and bottom portions of 

45 the contour, respectively. In other words, the branch points are points at which the correction is commenced. 

[0018] The height of each point of the edges of the book center can be obtained even if the book is placed on the 
document table in a slanting direction. However, when the image of the bound book is read by a line sensor, the book 
must be placed on the document table so that the directions of the top and bottom edges of the book are not parallel to 
the direction of the line sensor. Otherwise, when the image of the bound book is read by an area sensor, there is no 

so requirement regarding the direction of the book on the document table. 

[0019] In the image processing apparatus according to the second aspect of the present invention, the above 
image correcting unit may be a unit which corrects unevenness of lightness, deformation, or a blur of the image. 
[0020] When the image correcting unit in the second aspect of the present invention corrects unevenness of light- 
ness, the image correcting unit calculates, based on the heights of the respective points of the book center, corrected 

55 values of lightness for the respective pixels of the book center portion of the image, where the corrected values of light- 
ness; are values of lightness which the respective pixels of the book center portion of the image will have if the book 
center of the spread pages is in contact with the document table. Then, the image correcting unit changes the values 
of lightness which the respective pixels of the book center portion of the image, to the corrected values of lightness. 
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Thus, the unevenness of lightness of the image can be corrected. 

[0021 ] When the image correcting unit in the second aspect of the present invention corrects deformation, first, the 
image correcting unit obtains, by calculation based on the heights of the respective points of the book center, corrected 
pixel positions for the respective pixels of the book center portion of the image, where the corrected pixel positions are 
5 pixel positions at which the respective points of the book center of the spread pages will be located if the book center 
of the spread pages is in contact with the document table. Then, the image correcting unit moves the respective pixels 
of the book center portion of the image to the corrected pixel positions, and interpolated values are assigned to pixels 
which are vacated by the movement to the corrected pixel positions. Thus, the deformation in the book center portion 
can be corrected. 

10 [0022] When the image correcting unit in the second aspect of the present invention corrects a blur in the image 
obtained by scanning the book, the image correcting unit corrects the blur based on the heights of the respective points 
of the book center, by using a blur correction filter corresponding to a height (lift) of each point of the book center, i.e., 
a blur correction filter which is suitable for correcting a blur caused by the height (lift) of each point of the book center. 
[0023] When unevenness of lightness or a blur is corrected after deformation of the image is corrected, a value of 

15 the height for the interpolated pixel is necessary. In this case, the value of the height for the interpolated pixel can be 
obtained based on the heights of pixels located around the interpolated pixel. On the other hand, when deformation is 
corrected after unevenness of lightness and/or a blur is corrected, it is unnecessary to obtain a value of the height for 
the interpolated pixel. 

20 DESCRIPTION OF THE DRAWINGS 

[0024] 

Fig. 1 is a perspective view illustrating relative positions of a document table, a book document, and an image read- 
25 ing apparatus; 

Fig. 2 is a diagram illustrating an example of an image obtained by scanning a pair of spread pages of a bound book 
placed on a document table. 

Fig. 3 is a block diagram illustrating the construction of the image processing apparatus as the embodiment of the 
present invention; 

so Fig. 4 is a diagram illustrating an example of a binarized image of a pair of spread pages of a bound book placed 
on a document table; 

Fig. 5 is a diagram illustrating an example of a contour image of a pair of spread pages of a bound book placed on 
a document table; 

Fig. 6a is a simplified diagram illustrating a contour of a pair of spread pages of a bound book; 
35 Fig. 6b is an enlarged view of a book center portion of the contour of Fig. 6a; 

Fig. 7 is a diagram illustrating a geometrical relationship among a book document, a lens, and an image-forming 
plane; 

Fig. 8 is a diagram illustrating the upper area of a contour image of the spread pages for explaining the operation 
of the height distribution calculation unit 105; 
40 Fig. g is a side view illustrating a geometrical relationship among the spread pages of the book document 3, the 
document table 2, light sources 5a and 5b, a mirror 8, and a charge-coupled device (CCD) 9 in the flat bed scanner; 
Fig. 10 is a diagram illustrating a relationship between the angle OKyOand the height of each point of the surfaces 
of the spread pages of the book document 3; 

Fig. 11 is a diagram illustrating an example of a lift of the book center portion of the spread pages of the book doc- 
45 umerrt 3, and the movements of the respective points of the book center to the corrected positions on the document 
table 2; 

Fig. 12 is a diagram illustrating a way of obtaining the corrected point q n on the reference line; 
Fig. 13 is an enlarged view of the area 71 in Fig. 5; 

Fig. 14 is a diagram illustrating an example of distribution of vacated and unvacated pixels in an image generated 
so by the pixel movement in the area 71; and 

Fig. 1 5 is a diagram illustrating the fore edge portions of the image of the book document 3. 

DESCRIPTION OF PREFERRED EMBODIMENTS 

55 [0025] An image processing apparatus as an embodiment of the present invention is explained below with refer- 
ence to the drawings. 

[0026] The image processing apparatus as an embodiment of the present invention obtains a distance from a doc- 
ument table to each point of a book center portion of a pair of spread pages of a bound book placed on the document 
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table, and corrects deformation, unevenness of lightness, and a blur of an image of the bound book, based on the dis- 
tance. 

[0027] Fig. 3 is a block cfiagram illustrating the construction of the image processing apparatus as the embodiment 
of the present invention. As illustrated in Fig. 3, the image processing apparatus 100 as the embodiment of the present 

5 invention contains an analyzing unit 106 including a reference line extracting unit 101, a correction area determining 
unit 102, a distance calculation unit 103, an edge height calculation unit 104 and a height distribution calculation unit 
105. A lightness correction unit 107, a deformation correction unit 108 and a blur correction unit 109 are connected to 
the height distribution calculation unit 105. The lightness correction unit 107, the deformation correction unit 108 and 
the blur correction unit 109 are connected to a correction selecting unit 110, which is connected with a trimming unit 

w 111 that is connected to a skew correction unit 112. 

[0028] The analyzing unit 106 corresponds to the aforementioned height obtaining unit in the image processing 
apparatus according to the second aspect of the present invention. The analyzing unit 106 contains the reference fine 
extracting unit 101 , the correction area determining unit 102, the distance calculation unit 103, the edge height calcula- 
tion unit 104, and the height distribution calculation unit 105. which correspond to the reference line extracting unit, the 

is correction area determining unit, the distance obtaining unit, the edge height obtaining unit, and the height distribution 
obtaining unit, respectively. That is. the analyzing unit 106 obtains a plurality of heights, above the document table..of a 
plurality of points of the pair of spread pages of the bound book, based on a contour of the pair of spread pages which 
appears in the image of the book. 

[0029] Based on the height (distance) of each point of the book center portion above the document table, the light- 
20 ness correction unit 107 corrects unevenness of lightness in the image of the book, the deformation correction unit 108 
corrects deformation erf the image, and the blur correction unit 109 corrects a blur in the image. The correction selecting 
unit 1 10 performs necessary correction of degradation of the image by selectively using the lightness correction unit 
1 07. the deformation correction unit 1 08, and the blur correction unit 1 09. The trimming unit 1 1 1 trims off unnecessary 
portions of the image. The skew correction unit 112 corrects the skew of the image. 
25 [0030] Explanations of the respective elements in the construction of Fig. 3 are provided below. 

[1] First, explanations on the reference line extracting unit 101 are provided below. 

Fig. 1 shows a scene of scanning of a bound book by a flat bed scanner. In Fig. 1 , the bound book 3 is placed 
on a document table 2 of the flat bed scanner as an image reading apparatus 4. In Fig; 1 , the main scanning direc- 

30 tion, the sub-scanning direction, and the direction of the lift from the document table 2 are indicated by x, y, and z 
coordinates, respectively. Considering the aforementioned case in which the skew angles of the top and bottom 
portions of the contour of the pair of spread pages are different, four reference lines 1 1, 12, 13, and 14 (as illus- 
trated in Fig. 4) are extracted by the reference line extracting unit 101 in this embodiment. 

The image reading apparatus 4 contains a reduction optical system, and obtains density of each pixel of the 

35 image of the pair of spread pages in the form of digital value of n bits (e.g. 8 bits). In this embodiment, the digital 
value 255 corresponds to the pure white level, and the digital value 0 corresponds to the pure black level. Since, as 
illustrated in Fig. 2, the density of each point of the book center portion increases with the lift of the point, the digital 
value representing the density of each point of the book center portion decreases with the lift of the point. 

When an average density value in areas of the above image corresponding to portions of the spread pages 

40 which are in contact with the document table 2 is obtained, and the entire image is binarized based on the average 
density value, the binarized image as illustrated in Fig. 4 is obtained. In the binarized image of Fig. 4. the areas 91 
corresponding to the portions of the spread pages which are in contact with the document table 2 are white, and all 
of the other areas are black Next, almost straight portions of the top and bottom sides of the contours the areas 91 
are respectively extended to obtain four straight lines as the aforementioned four reference lines 1 1 to 14. In order 

45 to determine the straight lines, any known technique for detecting a straight line, such as the Hough transformation, 
boundary tracing, and the method of least squares, may be used. 
[2] Explanations on the correction area determining unit 102 are provided below. 

Fig. 5 is a diagram illustrating an example of a contour image of a pair of spread pages of a bound book placed 
on a document table. Since a book center portion of a scanned image such as the image of Fig. 2 is darter than 

so the aforementioned portions of the spread pages which are in contact with the document table 2, rt is impossible to 
extract contours of the book center portion of the spread pages from the binarized image such as the image of Fig. 
4. Therefore, the contour image of Fig. 5 is obtained by averaging the book center portion of a scanned image such 
as the image of Fig. 2, and then binarizing the averaged image. Thus, the contours of the book center portion of 
the spread pages can be extracted as illustrated in Fig. 5. 

55 in Fig. 5, the points 21, 22, 23, and 24, at which the reference lines 11,12, 13, and 14 branch from the top and 

bottom portions of the contour of the spread pages, are hereinafter called correction start points. Then, an area 
encircled by a straight line connecting the correction start points 2 1 and 23, a straight line connecting the correction 
start points 22 and 24, a portion of the contour of the spread pages between the correction start points 21 and 22, 
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and a portion of the contour of the spread pages between the correction start points 23 and 24 is defined as a cor- 
rection area 50. 

In addition, the points at which the contour is recessed most deeply are defined as book center points 31 and 
32, a line connecting the book center points 31 and 32 is defined as a parting line 41, and the area of Fig. 5 is 

5 divided by the parting line 41 into an upper area 61 and a lower area 62. 

[3] Explanations on the distance calculation unit 103 are provided below. 

Fig. 6a is a simplified diagram illustrating a contour of a pair of spread pages of a bound book, and Fig. 6b is 
an enlarged view of a book center portion of the contour of Fig. 6a. In Fig. 6b, Lp (n=1, 2. ... ) each denote a dis- 
tance from each point of the contour 10 of the spread pages to one of the reference lines 1 1 to 14. The distance Lp 

io is obtained for each pixel of a book center portion of the contour 10. In this case, for example, distances Lp and Lp. 
1 from two adjacent pixels to the reference line may become identical due to a quantization error. However, since 
the height changes gradually in the book center portion, a more accurate value of the distance Lp can be obtained 
by interpolation between the distances Lp^ and Lp^ . 
[4] Explanations on the edge height calculation unit 104 are provided below. 

15 Fig. 7 shows a geometrical relationship among a book document 3. a document table 2, a lens 6. and an 

image-forming plane 7. For example, when the book document 3 is not lifted from the document table 2, an image 
of a left edge of the book document is formed by light through the optical path D. Otherwise, when the book docu- 
ment 3 is lifted to the height H n above the document table 2, the image of the left edge of the book document is 
formed by light through the optical path E. In this case, there is a relationship between the height H n of each point 

20 of the left edge of the book document above the document table 2 and the aforementioned distance Lp from each 
pixel of the contour to the reference line, as expressed by the following equation (1). 

H n =BxA/(C-L n ). (1) 

25 where H n (n=1. 2. ... ) is the height of a point of an edge of the book document 3 above the document table 2, A isa 
distance between the point at which the optical path D intersects with the document table 2 and the point at which the 
optical path E intersects with the document table 2, B is a distance between the lens 6 and the image-forming surface 
(light-receiving surface) 7, C is a distance from the optical axis 42 to one of the reference lines 1 1 to 14, and Lp is the 
distance from each point of the contour 10 of the spread pages to the reference line. Thus, the height H n can be 
30 obtained from the distance Lp. 

[5] Explanations on the height distribution calculation unit 105 are provided below. 

The height distribution calculation unit 105 obtains by interpolation a height Hj n f (n=1, 2, ... ) of each point of 
the spread pages above the document table 2, based on the heights H n (n=1, 2, ... ) of the respective points of the 
edges of the spread pages above the document table 2. 
35 Since the image of the spread pages is divided by the parting line 41 into the upper area 61 and the lower area 

62, the following explanations are provided only on the upper area 61 with reference to Fig. 8, which shows the 
upper area of a contour image of the spread pages. 

A plurality of points p an (n=0, 1 , 2, ... ) ranging from the correction start point 21 (p a0 ) are defined on the left 
portion of the contour so that the plurality of points p an have y coordinates at intervals of Aa, and a plurality of points 
40 (n=0, 1 , 2, ... ) ranging from the correction start point 23 (Pbo) are defined on the right portion of the contour so 

that the plurality of points Pbp have y coordinates at intervals of Ab. When the difference between the y coordinates 
of the correction start point 21 and the book center point 31 is denoted by v, and the difference between the y coor- 
dinates of the correction start point 23 and the book center point 32 is denoted by w, Aa and Ab are set as follows. 

45 Aa=1 and Ab=w/v when w<v, 

Aa=v/w and Ab=1 when v<w, and 

Aa=Ab=1 when v=w. 

so 

For each value of n, the points p an and are connected by a straight line, and the height H in ' of a pant of the 
spread pages (of the book document 3) corresponding to each pixel Pj n (i=0, 1,2,...) located qq the straight line is 
obtained by one-dimensional interpolation between the height H an of the point p^ and the height H bn of the point 
Pbp. That is, the height H jn * can be obtained by the following equation (2). 

55 

H l p'=H an+ (H bn -H an )xx l /x J , (2) 
where Xj is the difference between the x coordinates of the points p^ and p^, and x^ is the difference between the x 
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coordinates of the points p an and p in . 

Furthermore, the accurate value of H in ' can be obtained by the following equation (2)' 



Hin'= Han +( Hbn-Han) x * x 1 (2)' 
*/x/ 2 +yy 2 



where yj is the difference between the y coordinates of the points p an and P bn , and y ( is the difference between 
the y coordinates of the points p an and p in . 

Thus, the analyzing unit 106 obtains a plurality of heights, above the document table, of a plurality of points of 
the pair of spread pages of the bound book, based on the contour of the pair of spread pages which appears in the 
image of the book. 

[6] Explanations on the lightness correction unit 1 07 are provided below. 

Fig. 9 is a side view illustrating a geometrical relationship among the spread pages of the book document 3. 
the document table 2, light sources 5a and 5b, a mirror 8, and a charge coupled device (CCD) 9 in the flat bed scan- 
ner. In the configuration of Fig. 9, the surfaces of the spread pages of the book document 3 are illuminated by light 
from the light sources 5a and 5b, and the light reflected by the point B on the surfaces of the spread pages of the 
book document 3 is lead through the point A on the document table 2 and is reflected by the mirror 8 to the charge 
coupled device (CCD) 9. In order to obtain a value of corrected lightness at the point A, intensities of the incident 
light and the reflection light at the points A and B are necessary, and can be obtained as follows. 

The intensities of the incident light at the points A and B are inversely proportional to the identical distance from 
the light sources 5a and 5b to the points A and B, respectively. 

That is, the intensity l A of the incident light at the points A is expressed as 

^*1/V( M/ n + «) 2 + r 2 (3) 

and the intensity l B of the incident light at the points B is expressed as 

l B *vJ( n / n + f2+H) 2 + r 2 (4) 

where f 1 is the thickness of the document table 2, f2 is the distance from the centers of the light sources 5a and 5b to 
the document table 2, n is the refraction index of the glass of the document table 2, r is the distance from the centers of 
the light sources 5a and 5b to the midpoint p between the centers of the light sources 5a and 5b, and H is the distance 
from the point B to the document table 2. 

The intensities of the reflection light at the points A and B can be obtained by the following equations (5) and 
(6), respectively. 

That is, the intensity R A of the reflection light at the points A is expressed as 

R A =l A xpx2cose, (5) 
and the intensity R B of the reflection light at the points B is expressed as 

R B =l B xpx(C0S\p1+C0S\Jf2), (6) 

where p is the reflectance of the paper of the book document 3, 6 is the angle at the point A between the incident light 
from the light source 5a and the reflection light, y1 is the angle at the point B between the incident light from the light 
source 5a and the normal to the surface, and is the angle at the point B between the incident light from the light 
source 5b and the normal to the surface. 

As illustrated in Fig. 9. cose is determined according to the positions of the light sources 5a and 5b, and can 
be obtained by the following equation (7). 

cos0=(f 1+f2)/V{r 2 + (f 1 + f2) 2 } (7) 

In addition, as illustrated in Fig. 9, y1 and y2 are expressed as 



y 1 =a+<P, 



EP1 032191 A2 



and 

y2=a-<I>. (9) 
Further, the angle a is obtained by the following equation (10). 

a«arctan(r/(f1+f2+H)) (10) 

The angle o> is obtained as follows. Fig. 10 shows a relationship between the angle o and the height of each 
point on the surfaces of the spread pages of the book document 3. 

As illustrated in Fig. 10, the height H(yi) of an arbitrary point C of the surfaces of the spread pages of the book 
document 3 can be obtained as an accumulation of tan(o>(y ,)) Ay in the section from the y coordinate y 0 of one of 
the correction start points 21 to 24 to the y coordinate yj of the point C. That is, 

i 

H(y,) = 5; tan(<*( yj ))x Ay (11) 
■ 2 tonWyj)) x Ay + tan(0>(y j)) x Ay 
= H(y M ) + tan(0(y ( ))x Ay. 

Si nce proc essing is performed on a digital image, Ay can be ^umed to i be one, i.e., Ay=1 , and ttierefore &(y-) 
can be obtained as 

<D(y ^arctanMy j)-H(y M )). (1 2) 

Since the reflection light received by a light receiving element can be deemed to be a ray from a point light 
source, the reflection density value at the point A is expressed as 

D A <*R A /a 2 . (13) 

and the reflection density value at the point B is expressed as 

D B ocR B /(a+H) 2 , (14) 

where a is a traveling length of the reflection light from the point A to the charge coupled device (CCD) 9. 

Since the proportionality constants in the relationships (13) and (14) are the same, the following equation can 
be obtained from the relationships (13) and (14). 

D A =D B x(a+H) 2 /a 2 xR A /R B (15) 

Based on the equations (5) and (6), the equation (15) can be rewritten as follows. 

D A =D B x ( a+ ^ 2/a2x2x ^ e /(^y 1 -w^v2)xl A /l B (16) 

Then, based on the equations (3), (4), and (16), the reflection density value D A of the point A can be obtained 
by the following equation (1 7). 

d a = D b x 2 x cos6/ ( cost|> 1 +cos>|i2 ) 
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D,=D B x 2 xcosS/ ( cbs\i>l+cost|>2 ) 



5 




(17) 



Therefore, the lightness at the point A can be obtained by substituting the aforementioned height H n or H in ' for 
H in the equation (1 7). Thus, the lightness at the point B can be corrected to the lightness at the point A. 
is [7] Explanations on the blur correction unit 1 09 are provided below. 

The blur in the book center image is caused by the lift of the book center portion of the spread pages from the 
document table 2. That is, due to the lift of the book center portion, the book center portion is located beyond the 
depth of field of the reduction optical system, and therefore a blur occurs. The degrees of blurs at the respective 
points of the book center image are different according to the lifts of the corresponding points of the book center 
20 portion of the spread pages. Therefore, the blur correction unit 109 corrects the blurs of the book center image by 
varying the amount of correction according to the degrees of the blurs. 

The degree of a blur is proportional to the height of each point of the book center portion. Therefore, image 
correction processing is performed by using a Laplacean operator so that the amount of correction varies with the 
of high frequency enhancement processing using a digital Laplacean operator. In the equation (18), f(i,j) indicates 
25 an image before correction, and g(i,j) indicates an image after correction. 

g(i.O=f(U>v 2 f(i.j) (18) 

However, in this embodiment a function of high frequency enhancement processing as expressed in the fal- 
se lowing equation (19) is used, where the digital Laplacean operator is weighted. 

g(iJ)=Hi.i)-r(H)xv 2 f(i,j) (19) 

The weight y(H) in the equation (19) is a function which is proportional to the height H. Therefore, when the height H 

35 increases, the weight j also increases. 

Thus, when substituting the aforementioned height H n or Hj n ' for H in the equation (19), correction of the blur 
can be performed so that the amount of the correction varies with the height H. 
[8] Explanations on the deformation correction unit 108 are provided below. 

The image correcting unit 1 08 obtains, by calculation based on the heights of the plurality of points of the book 

40 center portion, corrected pixel positions for the respective pixels of the book center portion of the image obtained 
by scanning the book document 3 placed on the document table 2, where the corrected pixel positions are imagi- 
nary pixel positions at which the respective points of the book center of the spread pages will be located if the book 
center portion of the spread pages Is in contact with the document table 2. Then, the image correcting unit 108 
moves the respective pixels of the book center portion of the image to the corrected pixel positions, and interpo- 

45 lated values are assigned to pixels which are vacated by the movement to the corrected pixel positions. Fig. 1 1 
shows an example of a lift of the book center portion of the spread pages of the book document 3, and the move- 
ments of the respective points of the book center portion to the corrected positions on the document table 2. 

First, explanations on correction of deformation in edges of the correction area corresponding to the edges of 
the book center portion are provided below. 

so It is assumed that, in the correction of deformation, a plurality of points r n (n=1 , 2, ... ) on one of the edges of 

the book center move to a plurality of corrected points q n (n=1 , 2, ... ) on one of the reference lines. The heights H n 
of the plurality of points r n (n=1, 2, ... ) can be obtained by the equation (1) based on the respective distances L„ 
(n=1 , 2, ... ) from the reference line to a plurality of points of the contour corresponding to the plurality of points r n 
(n=1,2,...). 

55 The corrected point q n on the reference line is determined as follows. Fig. 12 is provided for explaining a way 

of determining the corrected point q n on the reference line. In Fig. 12, the y* axis corresponds to the reference line. 

As illustrated in Fig. 12, the y* coordinate of the corrected point q n is determined by obtaining the correction 
amount k in the y* direction, i.e., in the direction of the reference line. In order to obtain the correction amount k, first, 
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the distances s 1 , s 2 , ... s n between adjacent points among the plurality of points r n (n=1 , 2, ... ) on one of the edges 
of the book center portion are obtained. Then, the increments t 1( t 2 , ... t„ of the / coordinate respectively corre- 
sponding to the distances s 1( $2, ... S n are obtained, as illustrated in Fig. 12. At this time, the distance s-, between 
the correction start point q 0 and the adjacent point ^ is obtained by the following equation (20). 

5 

•W0l 2 + H i 8 ) (») 

In addition, the distance s n between the point r n ^ and the adjacent point r n is obtained by the following equa- 
tion (21). 

10 ^ 

^n^VOn'+CHn-H^) 2 ) (21) 

Since the sum fo+s^ ... +s„) is approximately equal to the sum (^4*2+ • +t n +^. the above correction amount 
k can be determined by the following equation (22). 

M«i«b+ - +s n )-(ti+t 2 + ... +t n ) (22) 

Thus, the y' coordinate of the corrected point q n is determined. 

Although accuracy of correction is decreased, it is possible to obtain the correction amount k based on the dis- 
20 tance u between the correction start point q 0 and the point r n , instead of the sum (s^s^ ... +s„), for simplifying 
calculation. That is, 

k=u-(t 1+ t 2 +.;.+t n ). (23) 

25 AsdesCTlbed~ab^ 

mined, and the deformation in the edges of the correction area corresponding to the edges of the book center por- 
tion can be corrected by moving the pixels in the original image to the corrected points q n (n=1 , 2, ... ). 

Next, explanations on operations of correcting deformation of an image by moving pixels inside the edges of 
the correction area are provided below. 

30 Referring to Fig. 5, the aforementioned upper area 61 is divided by the optical axis 42 into the left area 71 and 

the right area 73, and the lower area 62 is divided by the optical axis 42 into the left area 72 and the right area 74, 
where the optical axis 42 is an optical axis of the reduction optical system in the scanner unit as illustrated in Fig. 7. 

In order to correct the positions of the pixels inside the edges of the correction area, it is necessary to obtain a 
reference line for each pixel. After the reference line is obtained, a corrected pixel position is obtained, and the pixel 

35 is moved to the corrected position, in a similar manner to the correction of deformation in the edges of the correc- 
tion area. For example, in the example of Fig. 13, a corrected pixel position q n ' corresponding to an uncorrected 
pixel position p n in the area 71 (in Fig. 5) is obtained as follows. 

Fig. 1 3 is an enlarged view of the area 71 in Fig. 5. In Fig. 1 3, the / axis corresponds to the reference line 1 1 , 
and the x' axis is in the cfirection perpendicular to the y' axis. First, a point q m \ which is located at a distance of 

40 from the uncorrected pixel position p n in the direction perpendicular to the reference line 1 1 , is obtained. The cfis- 
tance IV can be obtained from the relationship of the equation (1). In this case, the height H ln ' of the point p n is 
obtained from the equation (2), and is substituted for H n in the equation (1). In addition, the distance from the point 
p n to the optical axis 42 is obtained, and is substituted for the value (C-Ln) in the equation (1). The reference line n 
for the point p n is obtained as a straight line which passes through the above point q m ' and is parallel with the ref- 

45 erence line 1 1 . Thea a correction amount k' (which is similar to the correction amount k in Fig. 1 2) can be obtained 
by using the equations (20) to (22). Therefore, the point q n ' , which is located at the distance W from the point q m ' on 
the reference line n, is obtained as the above corrected pixel position for the uncorrected pixel position p n . Thus, 
the pixel at the point p n in the deformed image is moved to the point q n * in a corrected image. 

Due to the above movement some pixels in the corrected image may be vacated. Values (densities) which are 

so to be assigned to the vacated pixels are determined by interpolation. The interpolation is performed in each of the 
areas 71 to 74 jn Fig. 5 since the above movement of pixels in each of the areas 71 to 74 is performed based on 
one of the reference lines 1 1 to 14 corresponding to the area, and the distribution of the vacated pixels has the 
same directionality as the reference line corresponding to the area. Therefore, a density value which is to be 
assigned to a vacated pixel is determined by interpolation between uhvacated pixels existing in the direction of the 

55 reference line. 

Fig. 14 shows an example of distribution of vacated and unvacated pixels in an image generated by the pixel 
movement for correction of deformation in thie area 71 . For example, a density value to be assigned to a vacated 
pixel p m in Fig. 14 can be obtained as follows. 



10 



EP1 032191 A2 



First, an unvacated pixel near the vacated pixel p m is searched for in the direction of the reference iine 1 1 . 
When the unvacated pixel is searched for in the direction in which the y* coordinate increases, the unvacated pixel 
P2 is found. On the other hand, when the unvacated pixel is searched for in the direction in which the y* coordinate 
decreases, the unvacated pixel p 1 is found. When it is assumed that the densities of the unvacated pixels p 1 and 
P2 are respectively d1 and d2, the distance from the pixel p m to the pixel Pi is v1 , and the distance from the pixel 
p m to the pixel P 2 is w1 , the density value to be assigned to the pixel p m is determined by the following equation 
(24). 

dm=d1 xw1/(v1+w1)+d2xv1/(v1+w1) (24) 

When the deformation of the image is corrected as above, the image of the book center portion is extended. 
Therefore, in order to prevent overlap of the extended images of the spread pages, the extended images of the 
spread pages are relatively moved in the sub-scanning (y) direction. The amount of the relative movement of the 
extended images of the spread pages in the y direction is obtained based on the maximum amount of the move- 
ment of the pixels in the operation of correcting the deformation, i.e., the amount of movement of the book center 
points' 31 and 32. 

[9] Explanations on the trimming unit 1 1 1 are provided below. 

The trimming unit 1 1 1 detects and trims off the fore edge portions of the image of the book document 3. Fig. 
1 5 shows the fore edge portions of the image of the book document 3. The trimming unit 1 1 1 trims off the fore edge 
portions by extracting only an image of an innermost pair of pages (corresponding to the spread pages) in the 
image of the book document 3, as explained below. 

First, the image of the fore edge portions is binarized by using the high frequency enhancement processing. 
Then, the binarized image is divided into a plurality of areas, and it is determined whether or not a straight line 
exists in each of the plurality of areas. Since edges of the respective pages of the book document appear as a 
straight line in some areas which overlap the fore edge portions, the trimming unit 111 recognizes that the image 
of the desired pair of spread pages extends inside the innermost areas in which a straight line appears, and 
extracts only a region inside the innermost areas as the image of the desired pair of spread pages. In order to 
detect a straight line, the Hough transformation, boundary tracing, the method erf least squares, or the like may be 
used. 

[10] Other matters 

The book document 3 may be placed on the document table 2 in a slanting direction. Therefore, the image of 
the book document 3 may be skewed. Further, skew angles of the right and left pages may be different. In such 
cases, the appearance of an output image is deteriorated. Therefore, skew correction processing is performed by 
the skew correction unit 1 12. Even if the skew angles of the right and left pages are different, the skew correction 
unit 112 can obtain the skew angles based on the reference lines 11 to 14. The skew of the image of each page 
can be independently corrected by rotation processing such as the Affine transformation. 

[0031 ] As described above, in the above embodiment, correction of degradation of an image can be performed by 
only image processing, where the image is an image of a pair of spread pages of a book document 3 placed on a doc- 
ument table 2, and obtained by an image reading device having a reduction optical system. The degradation of the 
image may include unevenness of lightness, deformation, a blur, or the like, and occurs in a book center image. The 
correction is performed by only image processing, based on heights H n or H jn * of a plurality of points of the book center 
image. 

[0032] Although, in the described embodiment the operation of detecting the heights of the respective points of the 
book center portion is incorporated in the operation of correcting the degradation of the image, the operation of obtain- 
ing the heights per se can be used separately. 

[0033] All of the contents of the Japanese Patent Application No. 1 1(1999)-44965 are incorporated into this speci- 
fication by reference. 

Claims 

1 . An image processing apparatus comprising : 

a height obtaining unit which obtains a plurality of heights, above a document table, of a plurality of points of a 
pair of spread pages of a bound book as an original document, based on a contour of the pair of spread pages 
which appears in a scanned image of the pair of spread pages, where the bound book is placed on the docu- 
ment table; and 

an image correcting unit which corrects degradation of the image ol the pair of spread pages, based on the pi u- 
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rality of heights obtained by the height obtaining unit. 

2. An image processing apparatus according to claim 1 . wherein said height obtaining unit comprises, 

5 a reference line extracting unit which extracts at least one reference line based on a top and bottom portions 

of the contour of the pair of spread pages which appears in the image. 

a correction area determining unit which determines as a correction area an area of the image corresponding 
to a portion of the pair of spread pages which is lifted from the document table, based on said at least one ref- 
erence line and said contour, 

10 a distance obtaining unit which obtains a distance from each pixel of a portion of said contour in at least one 

edge of said correction area, to said at least one reference line, 

an edge height obtaining unit which obtains a height of each point of a top and bottom edges of said portion of 
the pair of spread pages above the document table, based on said distance obtained by the distance obtaining 
unit, and 

is a height distribution obtaining unit which obtains a height of each point of said portion of the pair of spread 

. pages above the document table, based on said height of each point of said top and bottom edges of said por- 
tion of the pair of spread pages. 

3. An image processing apparatus according to claim 1 , wherein said image correcting unit comprises a lightness cor- 
20 recting unit which corrects unevenness of lightness of said image. 

4. An image processing apparatus according to claim 1 , wherein said image correcting unit comprises a deformation 
correcting unit which corrects deformation of said image. 

25 5. Ah image processing apparatus according to claim i . wherein said image correcting unit comprises a blur correct- 
ing unit which corrects a blur of said image. 

6. An image processing method comprising the steps of: 

30 (a) obtaining a plurality of heights, above a document table, of a plurality of points of a pair of spread pages of 

a bound book as an original document, based on a contour of the pair of spread pages which appears in a 
scanned image of the pair of spread pages, where the bound book is placed on the document table; and 
(b) correcting degradation of the image of the pair of spread pages, based on the plurality of heights obtained 
by the height obtaining unit. 
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